Three instrumental means of minimizing the risks of technology are system verification, testing and maintenance. Every aspect of a computer system: hardware, software and communications, should be verified and thoroughly tested before the system is used for an electoral event. After successful testing, systems will need regular maintenance to ensure they will perform effectively when they are needed.
The importance of a technology application very likely determines the degree of rigour applied to verifying, testing and maintaining the technology. In the case, for instance, of a crucial electoral function, such as voting, all components of a voting system: hardware, software and communications should be regularly maintained and then verified and thoroughly tested before the system is used for an election.
System verification
For sensitive systems such as an electronic voting system, it is advisable to employ an independent testing office or organization to perform system verification tests, while for any other systems, proper verification and testing procedures can be developed and implemented in-house.
System verification tests or qualification tests may include:
- verifying that all system components namely, hardware, software and communications are capable of performing under expected normal conditions as well as under possible abnormal conditions, including if applicable, storage, transportation, operation and maintenance environments
- verifying that hardware conforms with local environmental requirements, including shelter, space, furnishings and fittings, electrical power supply and relevant extremes of temperature, humidity and pollution
- testing of hardware, software and communications to ensure that appropriate standards are followed and that they perform its intended functions
- performing audits of code
- revision of system documentation to ensure that it is adequate and complete
- testing system security measures to ensure that they are in place, that they are adequate and that they conform to appropriate standards
- verifying that appropriate quality assurance measures are in place
In addition, measures included in a software audit can include:
- verifying that the code is logically correct
- verifying that the programs follow a modular design, meaning that the code is made up of discreet programming modules that can be separately tested and evaluated
- verifying that there is no “hidden” code intended to perform unauthorised functions
- checking that the programming is straightforward, relatively easy to understand and contains code comments to facilitate maintenance by different staff
- verifying that the programming is designed to facilitate testing meaning that it includes code to allow testing of data flow of data within and between modules
- verifying that the code is robust including error treatment routines that prevent the loss of data while identifying, logging and reporting errors so as to allow for a rapid detection and correction of errors
- verifying that code incorporates security features that will prevent unauthorised access and/or detect and control any attempts at unauthorised access
- verifying that the system is user-friendly and does not require complex or obscure procedures that are difficult to follow
- verifying that the software can be easily installed in the live environment
- verifying that the software can be easily maintained, and that errors or defects can be easily identified, corrected and validated after installation
- checking whether the software can be easily modified to add new features
Once all the components of the system are verified, a report is issued and the necessary measures need to be taken to correct the problems found during the verification exercise. Once the corrections take place another round of verification needs to take place.
System testing
After a system has been verified, it needs to be thoroughly tested to ensure that every component of the system is performing in accordance with the specific requirements and that it is operating as it should including when the wrong functions are requested or the wrong data is introduced.
Testing measures consist of developing a set of test criteria either for the entire system or for specific hardware, software and communications components. For an important and sensitive system such as an electronic voting system, a structured system testing program may be established to ensure that all aspects of the system are thoroughly tested.
Testing measures that could be followed include:
- applying functional tests to determine whether the test criteria have been met
- applying qualitative assessments to determine whether the test criteria have been met
- conducting tests in “laboratory” conditions and conducting tests in a variety of “real life” conditions
- conducting tests over an extended period of time to ensure systems can perform consistently
- conducting “load tests”, simulating as close as possible likely conditions while using or exceeding the amounts of data that can be expected to be handled in an actual situation
Test measures for hardware may include:
§ applying “non-operating” tests to ensure that equipment can stand up to expected levels of physical handling
§ testing “hard wired” code in hardware (firmware) to ensure its logical correctness and that appropriate standards are followed
Tests for software components also include:
§ testing all programs to ensure its logical correctness and that appropriate design, development and implementation standards have been followed
§ conducting “load tests”, simulating as close as possible a variety of “real life” conditions using or exceeding the amounts of data that could be expected in an actual situation
§ verifying that integrity of data is maintained throughout its required manipulation
System maintenance
After systems have been verified, tested and implemented, they must continue to be maintained to ensure that they continue to perform correctly and that they can adapt to new requirements if needed.
Ongoing monitoring or testing of systems may need to be systematised to ensure that maintenance needs are identified and met when necessary. Where systems are for extended use, a mechanism can be put in place to monitor feedback from users as another means to determine the need for maintenance and modification.
Maintenance routines vary depending on the type and complexity of the technology. Many items come with a maintenance schedule or program recommended by the manufacturer or supplier. For some hardware and software, maintenance is provided by the manufacturer or supplier as part of the purchase agreement.
Where modifications to hardware, software and/or communications are made as a result of maintenance or upgrades, it may be necessary to conduct further rounds of system verification and testing to ensure that requirements meet the same or updated specifications.
